Heterogeneous Metric Learning for Cross-Modal Multimedia Retrieval

Due to the massive explosion of multimedia content on the web, users demand a new type of information retrieval, called cross-modal multimedia retrieval where users submit queries of one media type and get results of various other media types. Performing effective retrieval of heterogeneous multimedia content brings new challenges. One essential aspect of these challenges is to learn a heterogeneous metric between different types of multimedia objects. In this paper, we propose a Bayesian personalized ranking based heterogeneous metric learning (BPRHML) algorithm, which optimizes for correctly ranking the retrieval results. It uses pairwise preference constraints as training data and explicitly optimizes for preserving these constraints. To further encouraging the smoothness of learning results, we integrate graph regularization with Bayesian personalized ranking. The experimental results on two publicly available datasets show the effectiveness of our method.